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ABSTRACT 

This LISREL study examines the robustness of the 
maximum likelihood estimates under varying degrees of measurement 
model mis specification^ A true model containing five latent variables 
(two endogenous and three exogenous) and two indicator variables per 
latent variable was used. Measurement model misspecification 
considered included errors of omission, errors of inclusion, and 
simultaneous errors of omission and inclusion. A sample size of 200 
was used for all replications. LISREL programs were written for each 
of six models to be tested. The PC version of LISREL VII was used to 
estimate the models. Assessments of results were based on average 
parameter estimates for each model acrc^s replications ^ average 
chi-square value across replications, modification indices for errors 
of omission, and t-values for errors of inclusion. Results indicate 
that minor misspecifications in LISREL measurement models in the LY 
matrix are the most problematic, particularly for compound errors. In 
general, rhe ability of the LISREL program to detect measurement 
model misspecification is quite good. One figure illustrating the 
true population model and six data tables ar*? included. (TJH) 
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Researchers using LISREL or similar programs for the purpose 
of estimating the parameters of structural equation models must 
make important assumptions. Statistically, the researcher must 
assume large sample sizes (asymptotic theory) and a multivariate 
normal distribution of indicator variables in the measurement 
model* In addition, theoretical assumptions as to the appro- 
priateness of model specification are necessary • Unfortunately, 
usually one or more of these assumptions are violated. Much Monte 
Carlo research that has been done has been primarily concerned 
with non-normality and sample size (such as Boomsma, 1983; Gerb- 
ing & Anderson, 1985) . Other studies have been concerned with 
the use of categorical or dichotomous data (Ethington, 1985) , 
model misspecification (such as Gallini & Mandeville, 1983; Mac- 
Callum, 1986; Silvia & MacCalluro, 1988; Baldwin, 1987) and the 
test statistics used for model fitting (such as Kaplan, 1988) . 
This study examines the robustness of ML estimates under varying 
degrees of measurement model misspecification. 

Technicnies 

A true model containing five latent variables (two endogenous 
and three exogenous) and two indicator variables per latent vari- 
abJa was used (from KacCallum, 1986). Figure 1 illustrates the 
true model. Three types of measurement model misspecification 
were considered: (1) errors of omission, i.e., where a tro.e fac- 
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tor loading is omitted; (2) errors of inclusion, i.e., where an 
extra factor loading is added to the model; and (3) simultaneous 
errors of omission and inclusion. A sample size of N=200 was used 
for all replications. 

Using the assigned true parameter values, the model was spe- 
cified, and the population covariance Matrix genera' using the 
PROC IML program (SAS Institute, 1988) . The population covariance 
matrix S was used as input to the GENRAW and PRELIS programs 
{Joreskog & Sorbom, 1988) to generate 20 sample covariance mat- 
rices S for each model to be tested. Each variable was specified 
to be continuous with a mean of zero. LISREL programs were writ- 
ten for each of the six models to be tested. The PC version of 
LISREL VII (Joreskog & Sorbom, 1989) was used to estimate the 
models. 

The relevant output from the program runs consisted of the 
maximum likelihood parameter estimates, the standard errors of 
the estimates, t-values for all parameters being esti- 
mated, modification indices for all parameters that were not 
being estimated, and the chi-square goodness-of-f it valr.e with 
the associated degrees of freedom. 

Assessment of the results was based on the following crite- 
ria: 

(1) Average parameter estimates for each model across repli- 
cations (bias of sample estimates) : Does the average sample 
parameter estimate differ from the actual parameter value? 
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This relative difference was judged by computing a statistic 
which was the difference between the average parameter esti- 
mate and the parameter value divided by the parameter value 
and multiplied by 100. The relative difference represents the 
percentage increase or decrease in the value of the estimate 
as compared to the true parameter value. 

(2) Average chi-square value across replications: Would a 
misspecified model still be considered a good fit? What is 
the rate of rejection for misspecified models? 

(3) Modification indices for errors of omission 

A. Average modification index for a particular 
error. 

B. Percentage of cases in which the index is 
highest for the particular misspecification 
made: Does the modification index correctly 
indicate the adjustment to be made to yield a 
properly specified model? 

(4) T-values for errors of inclusion 

A. Average t-value for error: Is the t-value 
significant? 

B. Percentage of cases in which the t-value is 
insignificant: Does the t-value correctly 
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indicate that the misspecif ied parameter should 
be set equal to zero and thus yield a properly 
specified model? 

In addition to analyzing information about model misspecif ica- 
tion, parameter bias, and resultant goodness-of-fit, it is also 
of interest to consider the occurrences of improper solutions and 
nonconvergence in a Monte Carlo study. Improper solutions result 
when maximum likelihood estimates of variances are negative. 
These negative variances indicate that the solution is unstable. 
Nonconvergence was defined as the inability of the program to 
find a unique solution which meets the convergence criteria 
within 250 iterations* 

At the onset of this study, it was decided that improper 
solutions were to be included in the analysis, it was also 
decided that a minimum of 20 converged solutions was necessary 
for model analysis* 

Results 

Model lA. Model lA includes LX(6,2) as a factor leading. 
This error of inclusion means that the true model is contained 
within the misspecif ied model. The average parameter estimate for 
the misspecification was -.03 (tgs = -.16, p>.05). All other 
estimated parameters were significant. The only estimate that 
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substantially differed froia the parameter value was PS (1,1). The 
average chi-square value indicates that the model would be con- 
sidered plausible. The maximum modification indices were small 
and randomly scattered; thus, recovery from the misspecif ication 
would be expected. 

Model IB. Model IB omits LX(6, 3) - All estimated parameters 
were significant. GA(1,3) was underestimated due to the influence 
that the omission had on the latent variable. PH(3,3) indicates 
an increase in error variance. The maximum modification indices 
correctly indicated that LX(6,3) should be added to the model in 
every replication. The average chi-square value indicates that 
the model should not be considered plausible. Overall, deletion 
of a parameter as in this model appears to be more serious than 
inclusion as in Model lA. 

Model IC. Model IC includes LX(6,2) and omits LX(6,3). The 
t-value for the included estimate did not indicate that the par- 
ameter should be set to zero (t27 = 3.47, p<.05) . However, the 
modification indices would alert the researcher to the omitted 
element, yielding Model lA. Thus, using a two-step process, 
recovery from the misspecif icat ions would be likely. The misspec- 
if ication was correctly identified in evciry replication. As in 
Model IB, GA(1,3) was underestimated, and PH(3,3) was overesti- 
mated. The chi-square values were large for every replication, 
thus indicating that the model is not plaujible. 

ModeX 2A. Model 2A added LY(4,1) to the population model. As 
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for Model lA, all parameter loadings were significant except for 
the iflisspecif ication (t25 = .26, p>,05). The chi-square goodness- 
of-fit index indicates an acceptable model. PS (1,1) was under- 
estimated. 

Model 2B. LY(4,2) was omitted from the pc^: ilation model. 
Parameter es':imates for elements of GA, PS, TE, and TD were 
affected by the misspecj.f ication. PS (2, 2) was overestimated by 
112%, and TE(4,4) was overestimated by 212%. The chi-square val- 
ues indicate that the model is unacceptable, and the maximum mod- 
ification indices correctly flag the LY(4,2) factor loading in 19 
of the 20 replications. 

Model 2C. Model 2C simultaneously contains an error of inclu- 
sion (LY(4,1)) and an error of omission (LY(4,2)). Such errors on 
the "endogenous side" of the model presented many more estimation 
problems than for Model IC. Values for BE(2,1), GA(1,1), and 
PS(2,2) were overestimated, while values for GA(2,1), GA(2,2), 
and PS(1,1) were underestimated. T-values indicated that GA{2,2) 
should be deleted from the model, while LY(4,1) should be 
retained. In 16 of 20 replications, LY(4,2) had the highest modi- 
fication index. The average chi-square value is significant. Ove- 
rall, this model shows that measurement misspecif ication can 
seriously d^. tort the estimated values of structural coeffi- 
cients and fail to give the information necessary to correct the 
model • 
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Importance of the Study 

Questions as to the appropriateness of parameters 
derived from structural equation modeling under specific 
violations of assumptions have only been partially answered. 
Past research has concentrated on aspects of non-^normality 
and sample size with fewer studies investigating issues such 
as model misspecif ication, level of measurement, and 
goodness-of-f it. Researchers must be aware of the effects of 
violating assumptions when using sophisticated statistical 
programs such as LxSREL. This knowledge is essential to 
statisticians concerned with the development of covariance 
structure analysis as well as applied researchers in the 
field using structural equation modeling for theory 
development. 

This paper provides specific information as to the problems 
of measurement model misspecif ication for a model typical of 
those most often used in applied research. Overall, minor mis- 
specifications in LISREL measurement models in the LY matrix are 
the most problematic, particularly for compound errors, oven when 
the structur&l model is properly specified and the Scuuple size is 
adequate. In general, the ability of the LISREL program to detect 
measurement model misspecif ication is quite good. Thus, users 
can he relatively assured that program output (when combined with 
strong substantive theory supporting an adjustment) will aid them 
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in correcting a laisspecif icat.ion. 
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PARAMETER 


PARAMETER 


MEAN 


RELATIVE 


t 




VALUE 


ESTIMATE 


DIFF. 




LY 1,1 


1.0* 








LY 2,1 


.8 


.81 


— ** 


11.55 


LY 3,2 


1.0 








LY 4,2 


.8 


.75 




13.69 


LX 1,1 


1.0 








LX 2,1 


.8 


.79 




7.86 


LX 3,2 


1.0 








LX 4,2 


.8 


.83 




7.95 


LX 6,2 




-.03 




-.16 


LX 5,3 


1.0 








LX 6,3 


.8 


.81 




4.43 


BE 2,1 


.4 


.41 




2.82 


GA 1,1 


.4 


.40 




3.79 


GA 2,1 


.6 


.63 




4.32 


GA 1,2 


.6 


.58 




5.17 


GA 2,2 


.4 


.44 




2 .81 


GA 1,3 


.4 


.40 




3.52 


PH 1,1 


1.0 


1.01 




5.47 


PH 2,1 


.3 


.32 


— 


3.09 


PH 3,1 


.3 


.31 




3.26 


PH 2,2 


1.0 


1.01 




5.51 


PH 3,2 


.3 


.30 




2.78 


PH 3,3 


1.0 


1.06 




4 .16 


PS 1,1 


.5 


.43 


-14 


3.63 


PS 2,2 


.5 


.48 




3.65 


TE 1,1 


.6 


.63 




5.64 


TE 2,2 


.6 


.58 




6.91 


TE 3,3 


.6 


.60 




4.93 


TE 4,4 


.6 


.59 




6 . 66 


TD 1,1 


• 6 


. 61 




4 . 82 


TD 2,2 


.6 


.60 




6.70 


TD 3,3 


.6 


.58 




4.84 


TD 4,4 


.6 


.60 




6.46 


TD 5,5 


.6 


.54 




3.03 


TD 6,6 


.6 


.60 




4.58 



AVG MAX MI = 3.8 FOR LY 4,1 IN 3 OF 20 REPLICATIONS 

All others were smaller and randomly scattered. 

CHI-SQUARE = 23.51 with 25 df (p>.05) 

Number of converged replications =20 
Number of nonconvergent solutions = 0 
Number of improper solutions = 0 



* factor loadings of 1.0 are fixed for manifest variables 
** indicates a relative difference of 10% or less 



TABLE 1. RESULTS FOR MODEL lA 
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PARAMETER 


PAIIAMETER 


MEAN 


RELATIVE 


t 






VALUE 


ESTIMATE 


DIFF. 




LY 


1/1 


1.0* 








LY 


2,1 


.8 


.81 




11.33 


LY 


3,2 


1.0 








LY 


4,2 


.8 


.81 


— 


13.02 


LX 


1/1 


1.0 








LX 


2/1 


.8 


.81 


— 


7.48 


LX 


3,2 


1.0 








LX 


4,2 


.8 


.81 


— 


7.59 


LX 


5,3 


1.0 








BE 


2,1 


.4 


.43 


— 


3.26 


GA 


1/1 


.4 


.42 


— 


3.88 


6A 


2,1 


.6 


.63 


— 


4.37 


GA 


1/2 


.6 


.67 


12 


5.67 


6A 


2,2 


.4 


.39 


— 


2.32 


GA 


1,3 


.4 


.22 


-45 


3.39 


PH 


1/1 


1.0 


.97 


— 


5.24 


PH 


2,1 


.3 


.27 


— 


2.64 


PH 


3,1 


.3 


.30 


— 


2.86 


PH 


2,2 


1.0 


1.02 


- 


5.35 


PH 


3,2 


.3 


.29 


- 


2.73 


PK 


3,3 


1.0 


1.55 


55 


9.98 


PS 


1/1 


.5 


.57 


14 


4.16 


PS 


2,2 


.5 


.48 


— 


3.69 


TE 


1/1 


.6 


.57 


— 


5.02 


TE 


2,2 


.6 


.62 


— 


6.92 


TE 


3,3 


.6 


.61 


— 


4.97 


TE 


4,4 


.6 


.62 




6.60 


TD 


1/1 


.6 


.61 




4.88 


TD 


2,2 


.6 


.56 




6.07 


TD 


3,3 


.6 


.60 




4.76 


TD 


4,4 


.6 


.60 




6.35 


TD 


6,6 


.6 


1.23 




9.98 



AVG MAX MI = 64.2 FOR LX 6,3 IN 20 OF 20 REPLICATIONS 
CHI-SQUARE = 114.29 with 28 df (p<.05) 



Nximber of converged replications =20 
Number of nonconvergent solutions = 0 
Number of improper solutions = 0 



* factor loadings of 1.0 are fixed for manifest variables 
** indicates a relative difference of 10% or less 
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PARAMETER 


PARAMETER 


MEAN 


RELATIVE 


t 






VALUE 


ESTIMATE 


DIFF. 




LY 


1/1 


1.0* 








LY 


2,1 


.8 


.78 


-** 


11.34 


LY 


3,2 


1.0 








LY 


4,2 


.8 


.80 




13.45 


LX 


1/1 


1.0 








LX 


2,1 


.8 


.79 




7.87 


LX 


3,2 


1.0 








LX 


4,2 


.8 


.84 




7.93 


LX 


6,2 




.33 




3.47 


LX 


5,3 


1.0 








BE 


2,1 


.4 


.37 




2.69 


Gh 


1/1 


.4 


.44 




4.17 


GA 


2,1 


. 6 


. 63 




4. 63 


GA 


1,2 


.6 


.70 


17 


5. 64 


GA 


2,2 


.4 


.44 




2.55 


GA 


1,3 


.4 


.20 


-50 


3. 02 


PH 


1/1 


1.0 


1. 07 




5.50 


PH 


2,1 


.3 


.29 




2.91 


PH 


3,1 


.3 


.27 




2.52 


PH 


2,2 


1.0 


.96 




5.32 


PH 


3,2 


.3 


.36 


20 


3.33 


PH 


3,3 


1.0 


1.60 


60 


9.98 


PS 


1/1 


.5 


.57 


14 


4.09 


PS 


2,2 


.5 


.47 




3.68 


TE 


1/1 


.6 


. 58 




4.98 


TE 


2,2 


.6 


. 60 




6.95 


TE 


3,3 


.6 


. 60 




5.04 


TE 


4,4 


.6 


.57 




6.49 


TD 


1/1 


.6 


.92 


53 


4.75 


TD 


2,2 


.6 


.59 




6.46 


TD 


3,3 


.6 


.66 




5.55 


TD 


4,4 


.6 


.58 




9.73 


TD 


5,5 


.6 


.54 




3.03 


TD 


6,6 


.6 


.60 




4.58 



AVG MAX MI = 57.7 FOR LX 6,3 IN 20 OF 20 REPLICATIONS 
CHI-SQUARE = 94.26 with 27 df (p<.05) 



Number of converged replications = 20 
Number of nonconvergent solutions = 0 
Number of improper solutions = 0 



* factor loadings of 1.0 are fixed for manifest variables 
** indicates a relative difference of 10% or less 



TABLE 3. RESULTS FOR MODEL IC 
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PARAMETER 


PARAMETER 


MEAN 


RELATIVE 


t 






VALUE 


ESTIMATE 


DIFF. 




LY 


If 1 


1.0* 








LY 


2,1 


.8 


.81 


-** 


11.17 


LY 


4,1 


- 


.01 


- 


.26 


LY 


3,2 


1.0 








LY 


4,2 


.8 


.79 


- 


4.42 


LX 


1.1 


1.0 








LX 


2,1 


.8 


.80 


- 


8.02 


LX 


3,2 


1.0 








LX 


4,2 


.8 


.82 


- 


7.90 


LX 


6,3 


.8 


.81 


- 


5.55 


BE 


2/1 


.4 


.45 


- 


2.95 


6A 


1/1 


.4 


.39 


- 


3.96 


GA 


2,1 


.6 


.60 


- 


4.30 


GA 


1,2 


.6 


.60 


- 


5.41 


GA 


2,2 


.4 


.37 


- 


2.39 


GA 


1,3 


.4 


.40 


- 


3.53 


PH 


1/1 


1.0 


1.06 


- 


5.60 


PH 


2,1 


.3 


.31 


- 


3.03 


PH 


3,1 


.3 


.26 


- 


2.59 


PH 


2,2 


1.0 


1.00 


- 


5.51 


PH 


3,2 


.3 


.28 


- 


2.79 


PH 


3,3 


1.0 


.99 


- 


4.47 


PS 


1/1 


.5 


.44 


-12 


3.53 


PS 


2,2 


.5 


.52 


- 


2.80 


TE 


1/1 


.6 


.64 


- 


5.70 


TE 


2,2 


.6 


.57 


- 


7.09 


TE 


3,3 


.6 


.57 


- 


3.40 


TE 


4,4 


.6 


.58 


mm 


5.13 


TD 


1/1 


.6 


.56 


— 


4.62 


TD 


2,2 


.6 


.57 




6.36 


TD 


3,3 


.6 


.57 




4.79 


TD 


4,4 


.6 


.55 




6.21 


TD 


5,5 


.6 


.54 




3.36 


TD 


6,6 


.6 


.59 




4.98 



AVG M.\X MI = 4.0 FOR LX 1,3 IN 3 OF 20 REPLICATIONS 

All others were smaller and randomly scattered. 

CHI-SQUARE = 27.01 with 25 df (p>.05) ' 

Number of converged replications = 20 
Number of nonconvergent solutions = 0 
Number of improper solutions = 0 



* factor loadings of 1.0 are fixed for manifest variables 
** indicates a relative difference of 10% or less 



TABLE 4. RESULTS FOR MODEL 2A 
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PARAMETER PARAMETER MEAN RELATIVE t 

VALUE ESTIMATE DIFF. 



LY 1 


^1 


1.0* 








LY 2, 


^1 


.8 


.80 




11. 80 


LY 3 


f2 


1.0 








LX 1 


,1 


1.0 








LX 2 


^1 


".8 


.82 




7 . 15 


LX 3 


r2 


1. 0 








LX 4, 


r2 


.8 


.80 




7.95 


LX 6, 


r3 


.8 


.86 




6. 14 


BE 2 


^1 


,4 


.41 




2.72 


GA 1^ 


^1 


,4 


.37 




3 . 59 


GA 2 


rl 


. 6 


. 58 




4 . 05 


GA 1, 


f2 


. 6 


. 64 






GA 2, 


f2 


,4 


.41 




7 .49 


GA 1, 


r3 


.4 


.45 


12 . 5 


3 .85 


PH 1, 


,1 


1.0 


1. 01 




5 . 17 


PH 2, 


,1 


.3 


.27 




2 57 


PH 3] 


^1 


.3 


29 




2 ft4 


PH 2, 


2 


1.0 


1 04 






PH 3| 


2 


. 3 






3 00 


PH 3i 


3 


1.0 


.93 




4 fi3 


PS It 


1 


.5 


. 49 




3 .77 


PS 2, 


2 


. 5 


1. 06 


112 


7 . 99 


TE 1, 


1 


. 6 


. 57 




5.02 


TE 2, 


2 


.6 


.60 




6.83 


TE 4, 


4 


.6 


1,87 


212 


9.98 


TD 1| 


1 


.6 


.61 




4.51 


TD 2, 


2 


.6 


.60 




6.00 


TD 3, 


3 


.6 


.60 




4.86 


TD 4, 


4 


.6 


.58 




6.42 


TD 5, 


5 


.6 


.92 


53 


4.33 


TD 6, 


6 


.6 


.54 




4.79 



AVG MAX MI = 105.2 FOR LY 4.2 

LY 4,2 had the MMI in 19 of 20 replications. 

CHI -SQUARE = 196.87 with 28 df (p<.05) 

Number of converged replications =20 
Number of nonconvergent solutions = 0 
rumber of improper solutions = 0 



* factor loadings of 1.0 are fixed for manifest variables 
** indicates a relative difference of 10% or less 



TABLE 5. RESULTS FOR MODEL 2B 
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PARAMETER PARAMETER MEAN RELATIVE t 

VALUE ESTIMATE DIFF. 



LY 1,1 


1.0* 








LY 2,1 


.8 


.84 


-** 


10.94 


LY 4,1 


- 


.91 


— 


11.13 


LY 3,2 


1.0 








LX 1,1 


1.0 








LX 2,1 


.8 


.84 


— 


7.73 


LX 3,2 


1.0 








LX 4,2 


.8 


.78 


— 


7.50 


LX 6,3 


.8 


.77 


— 


5.98 


BE 2,1 


.4 


.84 


110 


3.98 


GA 1,1 


.4 


.54 


35 


5.21 


GA 2,1 


.6 


.29 


-52 


1.76 


GA 1,2 


.6 


.59 


— 


5.54 


GA 2,2 


.4 


.14 


-65 


.71 


GA 1,3 


.4 


.30 


-25 


3.41 


PH 1,1 


1.0 


.96 


— 


5.31 


PH 2,1 


.3 


.27 


— 


2.75 


PH 3,1 


.3 


.30 


— 


2.81 


PH 2,2 


1,0 


.97 


— 


5.33 


PH 3,2 


.3 


.31 


— 


2.85 


PH 3,3 


1.0 


1.15 


15 


4.78 


PS 1,1 


.5 


.25 


-50 


2.68 


PS 2,2 


.5 


.86 


72 


7.39 


TE 1,1 


.6 


.81 


35 


7.70 


TE 2,2 


.6 


.73 


22 


8.24 


TE 4,4 


.6 


.77 


28 


8.06 


TD 1,1 


.6 


.63 




5.21 


TD 2,2 


.6 


.59 




6.27 


TD 3,3 


- .6 


.59 




4.79 


TD 4,4 


.6 


.60 




6.70 


TD 5,5 


.6 


.56 




3.16 


TD 6,6 


.6 


.64 




5.33 



AVG MAX MI = 35.8 FOR LY 4,2 

LY 4,2 had the MMI in 16 of 20 replications. 

CHI-SQUARE = 71.62 with 27 df (p<.05) 

Number of converged replications =20 
Number of nonconvergent solutions = 0 
Number of improper solutions = o 



* factor loadings of 1.0 are fixea for manifest variables 
** indicates a relative difference of lo% or less 



TABLE 6. RESULTS FOR MODEL 2C 
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